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Abstract. We consider a self- attracting random walk in dimension d = 1, in presence 
of a field of strength s, which biases the walker toward a target site. We focus on the 
dynamic case (true reinforced random walk), where memory effects are implemented 
at each time step, differently from the static case, where memory effects are accounted 
for globally. We analyze in details the asymptotic long-time behavior of the walker 
through the main statistical quantities (e.g. distinct sites visited, end-to-end distance) 
and we discuss a possible mapping between such dynamic self-attracting model and 
the trapping problem for a simple random walk, in analogy with the static model. 
Moreover, we find that, for any s > 0, the random walk behavior switches to ballistic 
and that field effects always prevail on memory effects without any singularity, already 
in d = 1; this is in contrast with the behavior observed in the static model. 



PACS numbers: 05.40.Fb, 02.50. Cw, 02.50.Ey 



Submitted to: New J. Phys. 



The true reinforced random walk with bias 



2 



1. Introduction 

The reinforced random walk is an interesting example of stochastic process with memory, 
defined as the statistical problem of a random walker which is attracted towards the 
sites it has already visited. It represents the attractive counterpart of the well-known 
self-avoiding walk, which is biased away from the already visited region. Memory of this 
type is often present in physics, in ecology, in search strategies and in biological systems: 
in all these cases, the walkers modify the environment they move on in a local manner, 
by leaving signals, trails or concentrations of substances which bias their subsequent 
motion [U El El HI El El [7] . The process with memory can take place in physical space, 
like in bacterial motion, or on a theoretical network, as it happens in models of genetic 
mutations in sequence spaces [8l l9l [TO]. 

The memory effect of the true reinforced random walk is locally incorporated in 
the interaction between steps, and can be implemented applying several rules [2]. For 
instance, one can include the attractive part in the dynamics by increasing the weight 
for the jump probability towards a given site, according to the number of times the site 
has been visited [TT1 [T2] . In this case, the resulting self-attracting walk collapses in any 
finite dimension and, at long times, the walker is confined to move on a finite set of sites. 
A more interesting way to account for memory effects is by letting the jump probability 
to a given site be ~ exp(ku), where k = 1 for sites already visited at least once and 
k = for the others. This model, called the one-step (true) reinforced random walk, is 
related to the Donsker-Varadhan Wiener sausage problem [131 El US] and its properties 
have been studied in details in [121 HI El HI US]- In particular, it was shown that in 
high dimensional lattices (d > 2) it exhibits a non-trivial phase transition between a 
diffusive and a collapsed phase as a function of the parameter u. This is evidenced by, 
e.g. the behavior of the mean-squared displacement x t and of the mean number S t of 
distinct sites visited up to time t, which change in correspondence to a critical value u c . 
Indeed, St ~ t for u < u c (with a logarithmic correction for d = 2), but as for 
u > u c . In d = 1, no phase transition has been evidenced and St scales as in normal 
diffusion, S t ~ x t ~ y/t. 

Beyond such truly dynamical approach for modeling self-attracting random walks, 
a so-called static model has been defined. The static model - sometimes confused with 
its dynamic counterpart - actually describes the statistics of self-attracting chains: each 
possible path is considered as a particular configuration within the space the process is 
embedded in, and the weight of each instance is globally assigned according to the self- 
attracting interaction. As pointed out in the case of the self-avoiding walks [20], static 
and dynamical self-attracting random walks differ considerably and their long-time and 
critical behavior do not coincide. 

When a field biasing the walker toward a given direction and/or point is present, 
a few results have been obtained. For instance, the model with bias has been studied 
by pU |22l [23] in the case of static memory, evidencing the presence of a second (first) 
order phase transition in d — 1 (d > 2) as a function of the bias parameter, between a 
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ballistic regime, where the displacement x t scales linearly with time, x t /t — > v > 0, and 
a sub-ballistic regime, where x t ~ t a , with a < 1. These results have been obtained 
exploiting the mapping with a model of random walk diffusing on a medium displaying 
a uniform distribution of traps [2TJ [21] . However, this powerful mapping does not hold 
when memory effects are accounted for dynamically, so the true reinforced random walk 
still presents many open problems. 

Here we shall concentrate on the ID version of the true one-step reinforced random 
walk, accounting also for the presence of a bias; the bias is taken to point towards a given 
site called target. Indeed, while already exhibiting a non-trivial behaviour, the ID model 
is amenable to an analytical treatment for the main statistical quantities of interest 
in the dynamical process. Such results may be useful in deepening the considerable 
controversy and confusion generated by the reinforced random walk despite its simple 
definition. In particular, we consider the number of distinct sites visited after a time 
t and its growth rate, the mean displacement and the mean time spent on the border 
of the (instantaneous) span and we provide an analytic expression of their asymptotic 
behavior in very good agreement with extensive numerical simulations. As we will show, 
in the presence of a bias with strength s, the random walk behavior switches to ballistic, 

1. e. x t ~ S t ~ t, namely, in d = 1, field effects prevail on memory effects for any s > 0, 
without any singularity. This is in contrast with the behavior observed in the static 
model [21} [21]. Finally, exploiting the results obtained in the asymptotic region, we 
obtain a formal expression for the probability of realizing a given path, highlighting its 
explicit dependence on St, Xt and on the number of times the random walk has sojourned 
on the borders of the (instantaneous) spanned region. This allows us to properly extend 
to the dynamic model the mapping with the trapping problem, already exploited for 
the static model. 

The paper is organized as follows. In Sec. 2 we provide the reader with the main 
definitions and in Sec. 3 we describe the mapping with the trapping model. Then, 
we analyze in details the main statistical quantities, such as, in Sec. 4 the number of 
distinct sites visited, in Sec. 5 the time spent on the edges of the spanned region and in 
Sec. 6 the mean time to return to the target. Finally, Sec. 7 contains conclusions and 
Appendices include details on analytical calculations. 

2. Definitions 

Let Q = {V, E} be a graph with vertex set V and edge set E. Given i,j e V, the 
stochastic process we consider is defined by the following single-step jump probability 
from site % to j: 



with k t (j) = 1,0, according to whether j has or has not been visited before time t 
respectively, T(i) is the set of neighborhood sites of i, and u is a constant, whose sign 
distinguishes between reinforced (u > 0) or repulsive (u < 0) walks. We also account 




(1) 
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for the presence of a field biasing the random walk towards (s > 0) or away (s < 0) 
from a given site T G V, hereafter called target: 

e ukt(j) e -sdj 

^l ? ) = v P ttfc t (j)-«v jGrw (2) 

2-y e r(i) e 

where dj is the chemical distance between j and the target, i.e. the number of links 
between j and the position of the target, while s tunes the field effect. 

In the following we consider the case of a reinforced (it > 0) walk in the presence 
of an attractive target (s > 0), defined on a 1-dimensional lattice. When d — 1, the 
walker position at time t can be denoted by a scalar quantity Xt, and the site where the 
target is located is denoted by xt- By handling ([2]) we obtain the following probability 
to jump from x t to x t +\ = x t ±1 

exp[uk(x t ± 1) - s\x T - x t =F 1|] , . 

p{x t ± 1 \xt) = — , (3) 

where N t is the proper normalization factor and k dropped the redundant index t. 

Without loss of generality we can choose x = 0, so that x t represents the end-to-end 

distance of the walk at time t. 

For the analytical investigation of the process, it is convenient to focus separately on 

the initial transient regime (before eventually reaching the target) and the asymptotic 

regime (after reaching the target); these regimes correspond to a "longitudinal field" 

and to a "central field", respectively. In the former long as xt + 1 < xt, the 

position of the target can be ignored and the transition probability can be simplified as 

. , . . exp\uk(x t ± 1) ± s)] 
p(x t ±l\x t )= N -> ( 4 ) 

N t = e uk( ~ Xt+l ) +s -)_ e uk(x t -l)-s ^ ^ 

More generally, we can look at the process as a biased random walk moving on an 
inhomogeneous substrate characterized by three regions: at the interface between the 
visited and the unvisited region, the single step probability assumes different expressions. 
Referring to figure (PQ), where the single step probabilities to move rightwards in the 
spanned region and at interfaces are denoted by a, (3 and 7, we have 

e s 

a = r^r, (6) 

2cosh(s) V ; 

e -u/2+s 

13 = 2cosh(-u/2 + s)' (7) 

p u/2+s 

1 = • (8) 

' 2cosh(w/2 + s) v ; 



3. The mapping with trap models 



According to the model described in the previous section, we assign a probability p(j\i) 
to the walker to step from i to the neighbor site j during the evolution. As a result, 
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Figure 1. The biased reinforced random walk in d = 1 can be looked at as a biased 
random walk moving on an inhomogeneous substrate where the jump probabilities 
change according to the starting position of the particle: within the visited region, or 
on the left border, or on the right border. 



memory effects are accounted for at each time step, that is, normalization is local. 

Another common way of modeling reinforced (or repulsive) interaction between 
steps, is to assign to any possible realization w of the walk an energy E UjS (w) = 
uS(w) — sx(w), being S(w) the number of distinct sites visited and x(w) the end- 
to-end distance displayed by the walk w (here, again, u and s are parameters tuning 
the interaction). Then, the probability to realize a particular path is taken to scale 
exponentially with the related energy; the normalization is therefore global (see the next 
subsection for more details). Models with local and global normalization conditions are 
also referred to as dynamic and static models, respectively [18] . 

A comparative study of these models in one dimension [25] has shown that the 
emergent behavior depends crucially on the kind of normalization, either global or local. 
In fact, in the static model, by definition, all the paths sharing the same number of visited 
sites S (and, in the presence of bias, sharing also the same elongation x) have the same 
probability, conversely in the dynamic model paths with the same S (and x) can occur 
with different probability, depending on the number of times the walk visits the borders 
between different regions (see figure (pQ)), as will be explained in the following. 

Another point which distinguishes dynamic and static models is the possibility for 
the latter to be mapped into a trap model [231 E] ; a possible extension of this mapping 
to the dynamic case is discussed in Sec. 13.21 

3.1. The static model 

As anticipated, in the static model [HI [23] an energy is associated to any path w; in 
general the energy is E UyS {w) = uS(w) — sx(w), where u is the cost associated to a new 
visited site and s accounts for the field [j]. Accordingly, the probability for the path w 
of length t to occur is defined as: 




X In this model the field is intrinsically longitudinal i.e., it biases the random walk towards a given 
direction and the strength is constant. 
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with Z Ut8 (t) the partition function, given by 

Z UtS (t) = e- uS{w)+sx{w) = W{S, x, t) e - uS{w)+sx{w \ (9) 

w S,x 

where W(S, x, t) is the number of paths of length t with an elongation x and S distinct 
sites visited. 

Starting from the partition function, we can compute the mean number of distinct sites 
visited (S) t , the mean displacement (x) t and the mean square displacement (x 2 ) t as 

(S) t = --^hxZ u>s (t), (x) t = — In Z U:S (t), (x 2 ) t = (x) 2 t + — lnZ U)S (t), (10) 

Now, we recall that the survival probability for a 1-dimensional biased walk in the 
presence of a quenched concentration c of static traps after t step can be written as [21] 



P surv (t) = (2cosh S )-^(l - c )SW e -H (n) 

w 

so that, using and posing 1 — c = e~ u , we obtain 

P surv (t) = (2coshs)-% jS (t). (12) 



Of course, for the unbiased case, starting from pip , we recover the well-known 
Rosenstock approximation 

P surv (0 = 2-' £(1 - c) s ^ = ((1 - c) s ) t « (1 - c)< 5 >% (13) 

where the last relation, holding in the early-time regime and for c 1, i.e. u <C 1, 
allows the simple estimate Z u>s (t) ~ e~"^*. In general, through this mapping, the 
partition function describing the static model on the substrate Q at time t can be 
directly associated to the survival probability of a standard random walk on Q at time 
t, in presence of a concentration c = 1 — of static traps. 

^.I?. The Dynamical model 

Let us consider a particular path of t steps u> = {xo, x t }; exploiting (jlj) we can write 
the probability P u s (w) of realizing the path w in terms of macroscopic quantities, such 
as the number of distinct sites visited at time t and the end-to-end distance. In fact, 
we have 

t t 

Pu,s(w) = Y\_P( X i\ X i-l) = H eX P [ uk ( Xi ) + S ^& n ( X i - Xi-l)]/Ni 
i=l i=l 
t 



^-uSt+sxt I e ~ ut 



i=0 
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- 1+e 2s-u- 


tR 


~l + e~ u - 2s ~ 


t L 


1 + e 2s 




1 + e- 2s 





where we used Yll=i = t ~ &t- As to the denominator, posing k{xi + 1) = k + and 
k(xi — 1) = we obtain 

t t 

j J A , = JJexp[u(A; + + fc_)/2]2cosh[«(£; + - fc_)/2 + s]. (14) 

i=0 i=o 

This product can be decomposed in three parts, according to whether X{ corresponds to 
the right border {i e d B ), to the left border {i e 8l) or to a site within the span of the 
random walk (j 6 J), namely n!=o = iLed^ ' lLe,9 L ' ELe/- Considering the values that 
k + and k_ assume in the different regions, we obtain 
t 

Yl Ni = Jl e u 2 cosh s JJ e u/2 2 cosh(s - u/2) JJ e u/2 2 cosh(s + u/2). 

i=0 iel iGd R i£d L 

We call ti, tR, ti the number of times that the random walker has sojourned in any of 
the three regions, respectively Then, recalling that t = tj + t R + t L , the denominator 

gets 

t 

e -utJj N , = (2 cosh s) 

i=0 

= (2coshsy C(t R ,t L ). 

Therefore, given u and s, all the paths sharing the same span, elongation and border 
times, whose number shall be denoted with W(S,tB,t), are realized with the same 
probability 

ex V l-uS t + sx t ] 

KAw) - (2cosh S yc(t R ,t L y 

As a consequence, the probability of realizing any of these paths is P u>s (S,tB,t) = 
P u ,s{w)W(S,tB,t). We also define W(S,t) as the number of paths with span S and 
f(t B \S, t) as the relative fraction with t B visits on border sites, so that W(S,t B ,t) = 
f(t B \S,t)W(S,t). 

For simplicity, let us now focus on the unbiased case; this allows to neglect the 
quantity x t and (fT5l) simplifies into 

P u , s= o(w) = 2-V" s * (r^ps) B > ( 16 ) 

that is, the probability of a path in the dynamic model depends on the two stochastic 
(not-Markovian) variables St and t B = t R + t^. 

We now calculate the probability P u s=0 (S,t) of realizing any arbitrary path of 
length t which spans over S distinct sites, regardless of the number of times t B it visits 
a border. This can be obtained by summing P u , s =o{S,t B ,t) over t B , namely 

P u , s =o(S, t) = W(S, t) P u ,s=o(w)f(t B \S, t) = W(S, t) (P u , s=0 (w))s,f (17) 
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We note that W(S, t)2 1 can be looked at as the probability P u= o :S= o(S, t) that a simple 
random walk covers a span 5* in a time t. Thus, recalling f|T6|) . we have 



P 



u,s=0 



(S,t) 



-uS 



P u =o,s=o(S, t) 



1 + e~ u 



ts 1 



-u'S 



S,t 



where in the last passage we assumed f(t B \S, t) as a Poissonian with average (tB)s,t = 
2S, so that, exploiting the cumulant expansion, we get u' = u — 2(1 — e _u )/(l — e u ). As 
shown in Fig. [2J this picture is in agreement with numerical results at long times (with 
respect to S), while at shorter times time- dependent corrections have to be introduced. 




Figure 2. Main figure: fraction of paths f(ts\S,t) that after t — 1000 steps have 
stayed ts times on borders, given S = 30 (*), S = 60 (□), S = 80 (•) distinct visited 
sites, respectively. Data have been fitted by Poissonian distributions. Inset: natural 
logarithm of P u ,s=o(S, t)/ P u= o, s= o{S, t) for t = 5000 and the linear estimate — u'S (see 
Eq.QH). 

The expression in ([TBI can be looked at as the survival probability for a random 
walker which has visited 5* sites in the presence of a concentration c = 1 — exp(-u') of 
traps. In the limit u — > we get c = 0, while for u — > oo we get c = 1, as expected. 

note that in the static case we had as variables t and S, where the latter was 
canceled out via cumulants expansion. In the dynamic case we have ^ as additional 
variable, and it is analogously canceled out so that we are left with P UtS (S,t) to be 
compared with a survival probability of a walk where we specify both the length t and 
the span S. 



4. The number of distinct visited sites 



In this section, we estimate analytically the number of distinct sites visited, as a function 
of time. We recall that, restricting the analysis to the transient regime, the relative 
distance to the target can be neglected. First, we focus on the distribution P u<s (S,t), 
which was introduced in the previous section and provides the probability that, at time 
t, the number of distinct sites visited is S. We also define 
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• G U _ S (S, t), which is the probability that at time t the span is incremented from 5 — 1 
to S, being G(S,0) = 5 s ,u 

• F UjS (S,t), which is the probability that a walker, starting from a frontier node, is 
able to widen the span from S to 5" + 1 in exactly t time steps, 

• F UjS (S,t), which is the probability that a walker, starting from a frontier node of 
a span S, moves inwards and remains within the internal region for (at least) the 
following t time steps. 

Thus, the following relations hold (we drop subscripts u, s to lighten the notation) 

t 

p(S,t) = ^2G(S,k)F(S,t-k). (19) 

fc=0 

As for G(S,t), it satisfies the recursive equation 

t 

G(S + l,t) = ^G(S,t-k)F(S,k). (20) 

fc=0 

The two coupled equations can be treated within a generating-function formalism, being 
h(X) = ^i^(^)^*' the generating function of the arbitrary function h(t). As explained 
in Appendix A.l , when the right border can be neglected (e.g., in the presence of a 
strong bias or at long times), the particle is likely not to return to the left border, so 
that in F(S,t) we can drop the dependence on S, and we find 

1 - F(X) r ~ 1 s- 1 

mA)= TFM ' (21) 

where 

F(X) = * A - (22) 

2e 2s + e u_ e 2s+u + y/2e°+"y/l - 2X 2 + cosh(2s) 

is the generating function of the probability to extend the span on the right side. The 
final formula is obtained by plugging the last expression into (j2ip . By anti-transforming 
P(S, A) we get an estimate for P(S, t) which, as shown in figure (jSJ), is in good agreement 
with simulations as long as the bias is rather strong or t is large. Notice that, by choosing 
u = and s = 0, we get F(X) = A/[l + \/I — A 2 ], which consistently represents the 
generating function of the mean-first passage probability to a neighboring site for a 
simple random walk [26] . 

Starting from equation ( 12T1) . we can also obtain the generating function of the 
moments of the number of distinct sites visited, that is 

(S h ) x = ^5 fc P(5,A) = ^A*(5 fc ) t , 
s t 
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Figure 3. Upper panel: probability distribution P u ,s{S,i) calculated at time t = 70 
and t = 100. The agreement between simulation (symbols) and analytical estimate 
(from (|2Tj) . solid line) is good especially for large values of S. Lower panel: probability 
distribution P UjS (S,t) calculated at time t = 1000; data from simulations (symbols) 
and from the estimate provided by a Poissonian distribution with parameters taken 
from (EH and (l26l). 



In particular, for the first moment we obtain 

(s)x = J2 sp (s,\) = J2 sp w s ~ 1 

1 



d 
dF 



s 

F(\) 



F(X) 



1-A 



(24) 



1-A [1-F(A)](1-A) 
2e 2s A 



(1 - A) e 2s (-2 + e u + 2A) - e u - V^e s + U ^l - 2A 2 + cosh(2s) 

The previous equation can be handled out via Tauberian theorems (see e.g. |26j ) 
in order to infer the asymptotic behavior for (S) t - In fact, (S)\ can be restated as 
(S) x = (1 - A)- 2 L![1/(1 - A)], where, posing y = 1/(1 - A): 



Li(y) 



2e 2s - e u (e 2s - 1) + e u ^[e 2s - l) 2 + 4e 2s (2?/ - 1) /y 2 



2e 2s _ ye u( e 2s _ l) + ye u ^/(e 2s - l) 2 + Ae 2s (2y - l)/y 2 



turns out to be a slowly varying function such that when y — > oo, L(x) —> l/f(s,u), 
being f(s,u) = 1 + 2e"/(e 2s — 1). Hence, we have 



(S)i 



2tL l (t)+t 2 L' 1 {t) 
W) 



.,2a 



"I) 



3 2s 



l + 2e n 



t. 



(25) 



This result is checked numerically in figure (jSJ). 

Therefore, at long times, {S)t grows linearly with time, that is to say, the bias 
prevails against memory effects. This is consistent with [27], where the qualitative 
behavior of the ID random walk is found not to be affected by memory effects. More 
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precisely, the rate of growth for the number of distinct sites visited is just given by 
1/ f(s, u): it depends exponentially on u, without exhibiting any singularity. Indeed, for 
any s > the velocity turns out to be positive. This estimate can be compared with 
the result found in [2T] for the 1-dimension static model where they evidenced a phase 
transition, between a phase with a zero drift velocity v — for s < u and a ballistic 
phase with v = tanh(s — u) for s > u. The transition is argued to be of the second 
order, in the sense that v — > continuously and with a discontinuous second derivative. 
As for the second moment (S 2 )\, we have 

T-F(A)' 



<S 2 >a 



s 
d 2 



E 

s 



dF 2 



. s 

2F(\) 



F(X)[1-F(X)} 



A 



A) 



+ (5); 



(26) 



[1 — F(A)] 2 (1 — A) 

Again, we use Tauberian theorems to infer the asymptotic behaviour and we get 
(S 2 ) x - (S)x = (1 - A)~ 3 L 2 [1/(1 - A)], with L 2 (y) converging to 2L l (y) 2 , when y oo. 
We therefore get the asymptotic behaviour, {S 2 ) t ~ t 2 /[f(s, u)] 2 , which also suggests 
that in the limit of long t, the distribution for 5* is Poissonian. 



4-1- The linear growth rate of (S)t 

A better estimate of (S) t at small and intermediate times can be independently obtained 
by calculating the average time taken by the walker to increase the extent of the visited 
region. In this approach we still focus only on the movement of the right edge, neglecting 
the movement of the other border. More precisely, we compute the mean time taken to 
pass from a span of 5* sites to S + 1 sites. Referring to figure (TjJ, this corresponds to 
the mean first-passage time t(0 — > 1). This quantity enables us to determine the rate of 
growth for the asymptotic law of {S)t- 



1-a a l-(3 (3 



-10 1 

Visited region 

Figure 4. When the random walker is far enough from the left border, we can neglect 
the unvisited region on the left side and just focus on how the right frontier propagates. 

Starting from size 0, the random walk can either jump directly to site 1 with 
probability 0, or jump to site —1 with probability 1 — /3; a similar position can be 
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applied for the time £(— 1 — > 1) to go from site —1 to site 1, thus we have 

f (0 1) = + (1 - 0)[t(-l -+ 1) + 1], (27) 
-+ 1) = p [t(-l 0) + 1] + (1 - 0) [t(-l 1) + ->• 0) + $8) 

Posing r = £(— 1 — >• 0), we obtain 

*(-i -> i) = *(o i) = p + (i - p) ( + I | . 



The explicit expression for r is obtained in Appendix A. 2 as r = (tanhs) -1 , thus, 
substituting, we have 



t(0 -> 1) = 1 + e M 



tanh s 



-1 . 



(29) 



The mean waiting time can be related to the growth rate v of (S) t , namely d(S) t /dt 
v^t(0^ l)- 1 : 



1 



t(0 -> 1) 



1 + e" ( 



1 



V tanh s 



- 1 



~i -l 



,2s _ 1 



e 2s - 1 + 2c 1 



(30) 



Since £(0 — > 1) does not depend on time, we have (S) t ~ vt, hence recovering the same 
result of (1251) . obtained through the Tauberian theorem. 



CO 



-0.05 




- 0.5 



u = 1 










u = 0.1 



0.5 1 



1.5 2 

s 



2.5 3 3.5 4 



Figure 5. Growth rate for the number of distinct sites visited (S)t/t as a function of 
u (upper panel) and of s (lower panel) . Data from simulations (symbols) are compared 
with analytical estimates (see ([25]) and (1301) ). 



We can improve the estimate of t(0 — > 1) appearing in ( )2"9"|) . taking into account 
also the presence of the left border, assumed as reflecting; this assumption allows to 
get a lower bound for t(0 — > 1). In this case the time taken by the random walk to 
expand the visited region depends on the span length. Decomposing the first passage 
time in the same way as in ( 1271) and (1281) . the only difference being the inclusion of the 
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left border, the equations to solve are 

t(0 -> 1) = 0+ (1 -P)[t(-1 -> 1) + 1], (31) 
-> 1) = JJ(-l)|/9[t(_l — >• 0; L) + l] + (1 — -> 1) (32) 

nl r v 



0\L 



+ t(-l -+ 0; L) + 1] | + JJ(-l)|t(-l -> L; 0) + t(L ->• -1) + £(-1 ->• 

' L|0 ^ ^ 

*(L -1) = 7 | JJ (L + 1) [1 + t(L + 1 -> -1; L)] + (33) 
+ JJ (L + 1) [1 + t(£ -1)] I + (1 - T )|l + t(L - 1 L) + t(L 

Ll-l ^ ^ 



where L is the coordinate of left border set at —S (see Appendix A.ip , n^i?/" 2 ) * s the 
splitting probability of reaching x before y starting from z and t(z — » x; y) is the 
conditional mean exit time, i.e. the mean first-passage time to reach x from z without 
seeing y [28]. With some algebra one obtains 



*(0 ->• 1) = 1 + (e u + e 2s )(e" + e - 2s ) e - 2s e 



2s\ -2s -2(5-1) tanhs 



+ 



(34) 



+ e"~ 2s [l + coth(s) g{S, e tanhs )], 

where g(l,t) = [(t 2 ^ - 3)(t 2 ' - 1) + 2l{t 2 - l)]/[t(t 2 ^ - l)] 2 . We note that when 
s 1, the previous equation correctly gives £(0 — > 1) — )■ 1, while for S ^> 1 we recover 
the expression in (1301) . 

This analytical estimate has been compared with data from numerical simulations 
in Fig. ([6]): the asymptotic behavior is nicely recovered. Moreover, the expression in 
Eq. ( 1341 with respect to the one in Eq. ( l30i) . provides a better estimate at small times, 
yet recovering the very same asymptotic behavior at large times. 



150 





°0 0.5 1 1.5 2 

1 x10 4 

Figure 6. Average number of distinct visited sites (S)t'- comparison between data 
from simulations (symbols) and the analytical estimates where (S)t ~ t/t(0 — > 1), 
obtained neglecting the left border (see Eq. ([29|) . bright line) and including the left 
border (see Eq.([34|). dark line). Here u = 1 and s = 0.01. 
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5. Border times 



Let us now consider the average total time (t B )t spent on borders and let us highlight 
its dependence on (S) t - This relation was anticipated in Sec. I3.2[ as it was crucial 
to extend the mapping between the trapping problem and the reinforced walk to the 
dynamic case. 

Given a walk w, we define tj the number of times it stays on any strictly internal 
site and ti (t R ) the number of times that the process visits the left (right) border; of 
course, t B = t L + t R . 

Focusing on long times, from the definitions above and recalling Fig. ([[]) and Eq. ([6]) 
we can write the following relations 

t = ti + t R + tj, 

(S) t = (l- 1 )(t L ) t + ft(t R ) t , 

(x) t = (2 7 - \){t L ) t + (2/3 - \){t R ) t + {2a - l)t. 

We preliminary note that a positive field (s > 0) implies a > ft > 1 — 7 (see (jSj)) as 
well as (t R )t ^> (ti)t; therefore, we can write (S) t ~ (3(t R )t and (x) t ~ t, from which 
we can infer (t R ) t ~ t, namely the propagation is ballistic, which is consistent with the 
above results. On the other hand, in the absence of bias (s = 0), we get 1 — 7 = ft and 
a = 1/2, from which (S) t ~ (^s)t and (x t ) t ~ (ts)t, consistently with [29] . 

Now, we focus on the occupation of border sites, namely t R , and try to highlight its 
connection with (S)t- We consider the unbiased case, where the random walk has proba- 
bility 1/2 to move left/right within the span and probability 4> = exp(— u/2)/(2 coshw/2) 
to move outwards the span from a frontier site. 



Let us denote with ASt and At R .t the discrete differentials, given by 

AS t = S t +i - S t , At B = t B ,t+i - t B ,t- (35) 

Since both ASt and At Btt can assume value either or 1, the conditional probability 
P(ASt\At Bj t-i) can be written in the matricial form: 

^^)-(SI3SI3)-(i-*0- (36) 

Therefore, for the related average values we obtain 

(AS) t = J2 AS t J2 P(^tB,t-l)P(ASt\AtB,^x) = P(^tB,t-X = l)<f> = (f>(^B)t-X, 
ASt At Bi i_! 

from which a linear relation between (S)t and (t B )t-i follows: 

t-x t-2 

(3) t = ^(AS) k = ^(At B ) fc + (AS) t=0 = <P(t B ) t -i + 1. (37) 

fc=0 fc=0 
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In the bias-free case we measured numerically the joint probability P UtS (S,tB,t) 
that the walkers has stayed times on any border and that 5* distinct sites have been 
visited; results are represented in figure ((71). From this distribution we calculated {S) t , 
confirming (1371) . Of curse, for u = 0, we get = 1/2 as expected. 




600 800 1000 1200 1400 1600 

t B 



Figure 7. Joint probability P u s (S,tB,t) of having the number is of visits on borders 
and S distinct sites visited, measured in the absence of bias s = 0. Time is fixed at 
t = 100 and u = 1. Note that the plot is versus ts and to S — ts </>, in order to evidence 
the peak of the distribution. 



6. Target 

In this last section we analyze the behaviour of the random walk once the target has 
been reached; this corresponds to the case of a random walk in the presence of a "central 
field" pointing towards the point xt- For simplicity we assume that the span around 
the point xt is symmetric and we calculate the rate of growth for the number of distinct 
sites visited as well as the average time taken by the random walk to return to the 
target. 

6.1. Mean exit time and scaling law for (S)t 

After having reached the target, the single step probability follows the framework 
depicted in figure (jHJ). In order to calculate the mean time taken by a random walk 
to increase the span width, we need the mean waiting time to revisit the borders. As 
anticipated, we take the spanned region as symmetric with respect to the target; more 
precisely, to simplify the notation we take xt = and borders at — L and L, respectively. 
This condition simplifies the framework for the single step probability (see figure (|9])): 
we have one reflecting point in xt and a region characterized by a rightwards drift, where 
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Figure 8. The single step probabilities in the region around the target. In this scheme 
the RW is supposed to have visited a symmetric region around the target 
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Figure 9. The framework of figure ([8]) is mapped into a semi-infinite structure where, 
exploiting the symmetry, the target is placed at the origin and looked at as a reflective 
barrier. 



the probability for xt — > Xt + 1 is 1 — a, while at the border x = L, the probability to 
move in the right direction is 1 — 7. Following a procedure similar to the one presented 
in Section (14. ip . we can calculate the mean exit time, i.e the first passage time from L to 



L + l, namely t(L — > L + l). The details of the computation are shown in Appendix B.l 
leading to: 

t(L + 1 -> L + 2) = 1 + ^ - r e 2 (s+ itan M _ 1] . (38) 

1 — e 

From this result we can estimate the scaling law for {S) t - In fact, t(L — > L + 1) is 
the mean time the walker has to wait in order to increase the number of visited sites 
from S — 2L to 2L + 1. The time t(S) corresponding to S visited sites is: 



t(S) ~ J At(S) = J t(L -> L + I) 



(g3s+«— 2tanhs \ S f ^s+u \ 

2 sinhs tanhs ) ^ 2\ sinhs / u ' s, ^ 

where C is a finite constant depending on s and u. In Fig. ( 1101) we show that the 
expression in ( )39|) correctly estimates the leading behavior of (S) t - In particular, in the 
limit S — > 00, the first term in (|39|) provides the leading contribution, therefore, at long 
times we can retain only this term and invert the relation to get 



B(s,u) 



(40) 



with B(s,u) = exp(3s + u — 2tanhs)/(2 sinhs tanhs). In Fig. ( ITU]) we show the com- 
parison with simulations. The leading behaviour of (140]) is in good agreement with 
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Figure 10. Left panel: mean time t(S) to expand a span of width S laying 
symmetrical around the target site. Analytical estimates (see Eq. solid line) 

are compared with data from simulations (symbols). Right panel: Average number of 
distinct visited sites (S)t as a function of time. Analytical estimates (see Eq. (|40|) . solid 
line) are compared with data from simulations (symbols). Both panels show results 
referring to two different choices of parameter (u, s), as specified 



numerical data, the difference being due to our approximations (the symmetric span, 
the continue limit in ( 139]) ). Notice that the long time behaviour is dominated by the 
bias encoded by the factor (tanhs) -1 , while memory effects yield small corrections en- 
coded by the term B(s,u). 



6.2. Mean return time to the target 



Let us now consider the mean time to return to the target, denoted as Ty. Referring to 
figure (Q this is given by tt — 1 +t(l — >■ 0), namely the walker takes one step to move 
from xt = to 1 and t(l — > 0) to go from 1 back to 0. Assuming the span 2L, so that 
the distance between the target and the border is L, we can calculate t(l — > 0) fixing 
the distance of the border from the target at L, hence imposing a reflecting barrier at 
L + 1; this corresponds to a situation where u ^> 1 and/or L ^> 1. From calculations 



reported in |Appendix B.2 



t{l -+0) =t exi t(l) + 



n 



L\0 



exit 



L-l) 



2 

1 



1 



(41) 



Uo\l(L ~ 1) 

where t exit is the exit time from an interval L, starting from L — 1 and in presence of a 
bias towards 0. Therefore, the mean time taken by the random walk to return to the 
target site is 



TT = t(l 0) + 1 = 1 



tanh(s) 

e ±(L-l)tanh(s) _ 



L 



itanh(s) 



1 



gjLtanh(s) j 



+ 



3 2tanh(s) 



g2Ltanh(s) J 



^Ltanh(s) 



2e -2(L-l)tanh(s) ^ + e ^+u\ 
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In the limit for L — > oo, t? reaches the asymptotic value 
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(42) 



1 - e~ 2s 

Notice that r T can be related to the stationary probability for a biased random walker 




Figure 11. Stationary probability 1/tt for the biased walker to be on the target site 
as functions of the bias strength s. The comparison between data from numerical 
simulations (symbols) with fixed memory interaction u = 0.1 and the analytical 
estimate (solid line) given by ([42]) is very good, despite the assumptions of reflecting 
barrier on both borders of visited region. 



to be at the target site [30]. In fact, according to Kac's formula for irreducible graphs, 
there exists a unique stationary probability 7r and the mean number of steps needed to 
return to any point i, is 1/^. 



As shown in Appendix B.2 a random walk embedded in a finite chain of length L 
and moving according to the prescription ([3]) with u — 0, admits a stationary probability 
with 7r = (1 — e~ 2s )/[2(l — e~ 2s ( L ~^)]. Now, assuming L large, we obtain n — > 1/Vt- As 
expected, 7r is finite when s > and grows exponentially with s up to the asymptotic 
value 1/2, while for s — > 0, 7r is vanishing. 



7. Conclusions 

In this paper we studied analytically the ID one-step reinforced random walk: at each 
time step the probability for the random walk to jump to a given site is ~ exp(fcw), 
where k = 1 for the sites visited at least once and k = for the others; a positive value 
for the parameter u, ruling memory effects, ensures that the walk is self-attractive. 
Since memory effects are accounted for at each time step, this model is often referred to 
as "dynamic" or "true". Such models find applications in the investigation of systems 
where diffusing particles are able to change the environment as, e.g., in the evolution 
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of a surface of growing aggregates [31], in spatial exploration with learning [2] and 
angiogenesis [3] . In our analysis we also considered the presence of a field s biasing the 
particle towards a given direction/site. 

First, we highlighted some basic differences between the dynamic and the static 
model. In the latter case memory effects are accounted globally: the probability of 
realizing a given path w of length t scales exponentially with the number of distinct 
sites visited S and with its end-to-end distance x. The dynamic model is by far more 
difficult to treat, due to the fact that memory effects are accounted locally, hence yielding 
a complex non-Markovian dynamics. Indeed, we found an analytic expression for the 
probability P u ^ s (w) to realize a particular path w, for a given choice of parameters u and 
s, where we evidenced a dependence not only on S and x, but also on the number of 
times ts that the walker has sojourned on the border sites of the (instantaneous) span. 
By averaging P UjS (w) over all paths exhibiting the same span we found an expression 
which can be compared to the survival probability of a standard random walk with 
a given span, in the presence of a uniform concentration c of traps, hence extending 
the mapping already proved and far exploited in the static model. Interestingly, here c 
depends exponentially on u as well as on the walker "velocity" . 

We also obtained an expression for the probability P UjS (S,t) of visiting S distinct 
sites in a time t, in terms of generating functions and found analytically that {S) t ~ \/i 
in the absence of bias (recovering previous results [29J ) , and becomes ballistic with 
(S) t ~ t in the presence of bias, as soon as s > 0. Thus, memory effects induce second 
order corrections in this case. 

Finally, we studied the joint probability P UtS (S,tB,t), which for any given (large) 
S is peaked at te = S/<f), where <p is the probability to move outwards starting from an 
edge of the spanned region. 

All analytical results have been confirmed, at least in their leading terms, by 
numerical simulations. 
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Appendix A. Distinct sites visited 

Appendix A.l. Computation of the probability distribution P{S,t) 

In this section we present a procedure to calculate the generating function for the 
distribution P(S, t) of the number of distinct sites visited S up to time t. The latter 
can be formalized in terms of the probability G(S,t) that a random walker increases the 
span width from S — 1 to S at time t. Indeed, the following relation holds: 

t 

P(S,t)=J2G(S,k)F(S,t-k), 

k=0 
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where F(S, t) is the probability that a random walker, starting from a frontier site, does 
not broaden the span S after t steps. Now, G(S, t) satisfies the recursive equation 

t 

G(S+ l,t) = ^2G(S,t-k)F(S,k), (A.l) 

fc=0 

where F(S,t) is the probability that a random walker, starting from a frontier site, 
increases the span width from 5" to S + 1 in t steps. In terms of the generating functions, 
equation ( lA.lj) becomes: 



G(S + 1, A) = G(S, X)F(S, A). (A.2) 
The previous finite difference equation has solution 

5-1 5-1 

G(S, t) = Y[ F(R, X)G(S = 1, A) = Yl F{R, A), 

R=l R=l 

where we have considered that G(S = 1,T) = 5 fi0 , that is, G(S = 1, A) = 1. 

Due to the bias, the probability distribution for the position of the random walk 
will be peaked in the region close to the right edge and it will move away from the 
left edge, increasing the mutual distance. Hence, we can assume that the random walk 
never returns at the left edge, namely that this edge is fixed and consequently focus on 
the right side of the visited region. More precisely, we can fix the right border at x — 0, 
so that the visited region is the half-line of the negative x, the single step probability 
towards right on this region is a = exp(s)/2 cosh(s), while on the edge the probability 
becomes (3 = exp(s — w/2)/2cosh(s — u/2) in order to account for memory effects, see 
figure 0H). 

Within this framework the probability F(L, k) is independent of L, and can be 
restated as F(0 — > l,k), namely the first-passage probability from x = 0tox = linfc 
steps. 



Appendix A.2. Computation of F(0 — > 1, A) 

For the probability F(0 — > l,t) the following equations hold: 

F(0-H,t) =p6 1>t + (l-P)F(-l->l,t-l). (A.3) 

t 

F{-1 -> l,t) = -> 0,k)F(0 ->■ l,t — k). 

k=0 

In the former equation the first passage probability to reach 1 from is splitted in 
two terms according to whether the random walk moves directly to 1 or to -1, while 
in the latter equation the first-passage probability from -1 to 1, is decomposed in two 
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processes: the random walk first reaches 0, then from it reaches 1. The related 
generating functions are 

F(0-H,A) =p\ + {l-/3)F(-l -> 1,A), (A.4) 

F(-l ->• 1, A) = F(-l -> 0, A)F(0 -> 1, A). (A.5) 

By plugging equation (1A.4I) into flA.5j) we obtain 

A/3 



F(0 -> 1, A) 



1 - A(1-/3)F(-1 -)• 0, A) 



The advantage of this formulation is that now we have to find F(—l — > 0, A), which 
involves a simple random walk in the presence of bias; this can be found via standard 
techniques (see e.g. as 



F(-l 0, A) = eS(c °^ h(s)) (l - y/1 - A'sech(s)' 
With this result we finally get 

2e 2s X 

F(0-»1,A) 



2e 2s + e u_ e 2s+u + y^es+u^/i _ + cosh(2s) ' 



(A.6) 



In the main text F(0 — > 1,A) is denoted as F(X) to lighten notation. Now, we can 

obtain an expression for the probability distribution of the number of distinct sites 

-| s-i 



visited P(S,t). The probability G(S, A) is simply 



F(-l ->■ 0,t) 



so that 



P{3, A) = F(0 -)• 1, A) [F(0 -> 1, A)] 



5-1 



(A.7) 



The probability F(0 — > 1,A) can be obtained from the first-passage probability 
F(0 ->■ 1, A) as 

F(0 -> 1, A) 



1- A 



1-F(0^1,A) , 



which plugged into (1A.7j) yields: 



P(S,\) 



1 - F(0 -» 1, A) 

t^a 



F(0 -)> 1, A) 



i s-i 



(A.8) 



The final formula is obtained using (1A.6j) in (1A.8j) . 



Appendix B. Target 

Appendix B.l. Computation of the mean passage time from L to L + 1 



Following a procedure similar to the one presented in Appendix A. 2 we calculate the 
mean exit time, i.e the first passage time from L to L + 1, denoted as t(L — > L + 1). 
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Referring to figure (7), we can write 

t(L ->■ L + 1) = (1 - 7) + 7 (t(L - 1 L + 1) + 1), (B.l) 
t(L - 1 -> L + 1) = - 1) [(1 - 7) (t(L - 1 ->• L; 0) + 1) + (B.2) 

L|0 

+ 7 (t(L - 1 L; + 1) + t(L - 1 L + 1)] + 

+ Y[(L - 1) [t(L - 1 -> 0; L) + t(0 ->■ L - 1) + t(L - 1 -> L + 1)] , 

where n^O 2 ) means the splitting probability of reaching x before y starting from z and 
t(z — > x; y) is the conditional mean exit time i.e. the mean first-passage time to reach 
x from z without seeing y [28]. After some algebra ( 1B.2j) becomes 

-l 



t(L- 1 ->L + 1) 



x 



(B.3) 



1-7 - !) - IE L - !) 

L|0 0|L 

| - 1 -)> L; 0) + 1] + n(!) [ f ( L - 1 0; L) + 1(0 ->• L — 1)] 1. 

^ L|0 0|L J 

Assuming x = reflection point, we can write 

t(0 ->■ L- 1) = 1 +t(l ->■ L- 1), 
*(1 ->£-!) = JJ (i) [ t (l^L-l;0) 

L-1|0 

+ J [ (1) [t(l 0; L - 1) + 1(0 ->• L - 1) 



(B.4) 
(B.5) 



0IL-1 



Substituting (1B.5|) in ( 1B.4|) we obtain 

1 + EL-HOW t(l^L- 1; 0) + rioiL-i(l) *(1 0; L - 1] 



t(0 — > L — 1) 



Using the last expression in ( IB. 31) we have 



i-n 



0|L-1^ 



£(L-1 ->L + 1) 



1-7 - !) - n( L - ^ 



-1 -1 



L|0 



OIL 



X 



(B.6) 



| " !) - 1 £; 0) + 1] + - !) — 1 — ^ 0; L) + t(0 — » L — 1)] j. 

^ L|0 0|L J 

What we need in the last expression are the conditional exit times, i.e. t(l — > L — 1; 0) 
and t(l — > 0; L — 1), but these involve only a simple random walk with bias and are 
known from the literature |32j. After some algebra we obtain 

cosh(s + u/2) 



t(L - 1 -> L + 1) = exp tx/2 - 2s - 2(L - 1) tanhs 



sinh s 



The last expression has to be substituted in (1B.1I) to obtain the mean exit time from a 
support of 2L distinct visited sites 

2e u 



t(L + 1 ->■ L + 2) = 1 + 



1-e 



-2s 



g2(s+L tanhs) j 



(B.7) 
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Appendix B.2. Computation of the mean return time to the target 



Here, we derive the mean time to return to the target: looking at figure ([9]), this requires 
one step (from site to site 1) together with the mean number of steps to return in 
starting from 1. Hence, we distinguish between the paths which reach the border L 
before returning to the target and those that do not. Accordingly, we have 

t(l->0) = t(l-+0;L)+n(l) 

0\L L\0 



t(l L;0)+t(L -»■ 0) 



(B.8) 



t(L ->• 0) = 7 



t(L - 1 -> 0) + 1 



+ (1"7) 



t(L -»■ 0) + 2 



(B.9) 



The last equation accounts for a reflecting barrier at L + 1, so that the walk reaches 
L + 1 and at the next step it returns to L with probability 1. The closure of the system 
is due by the following equation for t(L — 1 — > 0): 



t[L-\ -> 0;L) + 1 



0|L 

which plugged in the equation flB.9j) . after some algebra, gives 



-IK*- 1 : 



LIO 



t(L-l -> L;0)+t(L -> 0) 



7 



t(L -»■ 0) 



rioiL^ - 1) t(L - 1 0; L) + rU(£ - 1) t(L - 1 L; 0) 



+ 2-7 



7n iL^-i) 



Now, the expression in the square bracket is the unconditional exit time from an interval 
L in presence of a bias towards starting from L — 1, namely t exit (L — 1). Substituting 
in equation (1B.8|) we obtain the following expression: 



ta -» o) = Ut(i) + n^ 1 ) ^ l °) = + 



n 



LIO^ 



L]0 



n |L^-i) 



t exit (L - 1) + - + 1 

7 



By using the explicit expression for the splitting probabilities found in [2H], we 
obtain 



r T = t(l ->• 0) + 1 = 1 
L-l-L 



1 



tanh(s 

e i(i-l)tanh(s) _ ]_ 



C2 



^tanh(s) 



C2 



^Ltanh(s) 



+ 



3 2tanh(s) 



p2Ltanh(s) 



X 



2e -2(L-l)tanh(s) ^ + £ 2s+u\ _ 



(B.10) 



g2Ltanh(s) j 

This time can be related to the stationary probability of a random walk moving on a 
finite chain of length L + 1 and with probability to move leftwards (rightwards) given 
by p_ = e s I [e s + e~ s ) (p_ = 1 — p + ). The overall process is described by the tridiagonal 
transition matrix 





T 



p_ 

1 
p + 

•■■ 

■•• 



P- 



P+ o 
p + 







1 





(B.ll) 
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where we properly accounted for boundary conditions. The stationary probability 
7r can be looked at as the eigenvector satisfying Tn = ir, with J2i=o n i = ^ an< ^ 
< 7Tj < l,Vi G [0, L}. One finds 

7T. = ne- 23 ^, i G [1,L- 1] (B.12) 
Tr =jre s /(e s + e~ s ) (B.13) 
n L = ne- 2 < L -Ve- s /(e s + e" s ), (B.14) 

with the normalization factor n = [1 — e~ 2s ]/[2e s [l — e _2s ( L_1 )]]. In particular, 

1 - e- 2s 
710 ~ 2[1 -e-2*(i-i)]' 
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